#Meta FAIR

'Meta FAIR released CWM, a 32B dense decoder-only LLM mid-trained on Python execution traces and agent interactions to teach semantics and long-horizon reasoning for code generation.'

Meta and collaborators developed a novel framework to accurately quantify how much language models memorize training data, estimating GPT models store around 3.6 bits per parameter and providing new insights into memorization versus generalization.